Upgrade to snowflake-connector-python 4.0.0 and set cert lock timeout to prevent deadlock #1368
Conversation
|
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR. CLA has not been signed by users: @wiggzz |
|
Thank you for your pull request! We could not find a changelog entry for this change in the dbt-snowflake package. For details on how to document a change, see the Contributing Guide. |
|
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR. CLA has not been signed by users: @wiggzz |
|
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR. CLA has not been signed by users: @wiggzz |
|
Actually, looks like snowflakedb/snowflake-connector-python@b301717 is in |
Ok the new snowflake connector version fix is in but it requires a specific timeout parameter to be set. I'll see how difficult it is to upgrade to 4.0.0 and go from there. |
wiggzz
force-pushed
the
wj/snowflake-deadlock-client-prefetch
branch
from
11fa36d to
49d318e
Compare
wiggzz
changed the title
MichelleArk
temporarily deployed
to
dbt-snowflake
GitHub Actions
Inactive
| insecure_mode=creds.insecure_mode, | ||
| platform_detection_timeout_seconds=creds.platform_detection_timeout_seconds, | ||
| session_parameters=session_parameters, | ||
| ocsp_root_certs_dict_lock_timeout=10, # cert lock can cause deadlock without timeout, see https://github.com/snowflakedb/snowflake-connector-python/issues/2213 |
There was a problem hiding this comment.
is there a reasoning behind 10? Can we make this a user configurable value?
There was a problem hiding this comment.
My surface level understanding is that this is the amount of time that the connector will wait to obtain a lock to interact with some certs it uses during the request. I doubt that this will ever take longer than 10 seconds unless there is a deadlock, in which case we'll fail after 10 seconds. So I doubt there is a reason a user would care or want to configure it, I am personally a little confused why the connector exposes this at all. I think this should have been an implementation detail. But I may be misreading it, I'll try to take a deeper look tomorrow.
There was a problem hiding this comment.
Thanks, we can go with that for now and do a fast follow to make this configurable
colin-rogers-dbt
had a problem deploying
to
dbt-snowflake
GitHub Actions
Failure
Fix snowflakedb/snowflake-connector-python#2213
Problem
Snowflake queries can hang at exit
Solution
Upstream snowflake-connector-python added a fix for the hang which is to add a timeout to the certificate lock (which was deadlocking). This comes in snowflake-connector-python 4.0.0 and requires the explicit use of a timeout.
Checklist